Enhanced diagnostic accuracy for quantitative bone scan using an artificial neural network system: a Japanese multi-center database project

BACKGROUND Artificial neural network (ANN)-based bone scan index (BSI), a marker of the amount of bone metastasis, has been shown to enhance diagnostic accuracy and reproducibility but is potentially affected by training databases. The aims of this study were to revise the software using a large number of Japanese databases and to validate its diagnostic accuracy compared with the original Swedish training database. METHODS The BSI was calculated with EXINIbone (EB; EXINI Diagnostics) using the Swedish training database (n = 789). The software using Japanese training databases from a single institution (BONENAVI version 1, BN1, n = 904) and the revised version from nine institutions (version 2, BN2, n = 1,532) were compared. The diagnostic accuracy was validated with another 503 multi-center bone scans including patients with prostate (n = 207), breast (n = 166), and other cancer types. The ANN value (probability of abnormality) and BSI were calculated. Receiver operating characteristic (ROC) and net reclassification improvement (NRI) analyses were performed. RESULTS The ROC analysis based on the ANN value showed significant improvement from EB to BN1 and BN2. In men (n = 296), the area under the curve (AUC) was 0.877 for EB, 0.912 for BN1 (p = not significant (ns) vs. EB) and 0.934 for BN2 (p = 0.007 vs. EB). In women (n = 207), the AUC was 0.831 for EB, 0.910 for BN1 (p = 0.016 vs. EB), and 0.932 for BN2 (p < 0.0001 vs. EB). The optimum sensitivity and specificity based on BN2 was 90% and 84% for men and 93% and 85% for women. In patients with prostate cancer, the AUC was equally high with EB, BN1, and BN2 (0.939, 0.949, and 0.957, p = ns). In patients with breast cancer, the AUC was improved from EB (0.847) to BN1 (0.910, p = ns) and BN2 (0.924, p = 0.039). The NRI using ANN between EB and BN1 was 17.7% (p = 0.0042), and that between EB and BN2 was 29.6% (p < 0.0001). With respect to BSI, the NRI analysis showed downward reclassification with total NRI of 31.9% ( p < 0.0001). CONCLUSION In the software for calculating BSI, the multi-institutional database significantly improved identification of bone metastasis compared with the original database, indicating the importance of a sufficient number of training databases including various types of cancers.